WEAKLY DEPENDENT CHAINS WITH INFINITE MEMORY 



PAUL DOUKHAN (1) ' (2) AND OLIVIER WINTENBERGER (2) 

Abstract. We prove the existence of a weakly dependent strictly stationary solution of the 
equation Xt = F(Xt-i, Xt-2, Xts, . . . ; £t) called chain with infinite memory. Here the inno- 
vations £t constitute an independent and identically distributed sequence of random variables. 
The function F takes values in some Banach space and satisfies a Lipschitz-type condition. We 
also study the interplay between the existence of moments and the rate of decay of the Lipschitz 
coefficients of the function F. With the help of the weak dependence properties, we derive Strong 
Laws of Large Number, a Central Limit Theorem and a Strong Invariance Principle. 



1. Introduction 

Statistical inferences heavily rely on the underlying model. Processes may have different repre- 
sentations. Thus they belong to different classes of models. In this paper, we introduce a chain 
with infinite memory as the stationary solution of the equation 

(1.1) X t = F{X t - lt X^2,X t -. 3t ...;Z t ), a.s. for t € Z, 

where F takes value in a Banach space. For details, see section 13.11 The dynamical behavior 
described by (jl.lB corresponds to a large variety of times series models. Those models can be seen 
as natural extensions, either of linear models or of Markov models. In the sequel, the innovations 
£t constitute an independent and identically distributed (iid) sequence. Various representations 
use such innovations. For instance, the case of causal Bernoulli shifts X t = H(£tj£t—lj • • •) was 
studied by Wu [31]. But several Bernoulli shifts, such as Volterra series, may not fit the parsimony 
criterion and the function H may be non-explicit. This is a drawback for statistical inferences in 
that context. Markov models are preferred in various applications e.g. in finance, hydrodynamics, 
physics, see [111125). Kallenberg [23] stresses the fact that all the p-Markov processes are solutions 
of equations of the type: 

(1.2) X t = F(X t - 1 ,...,Xt- p ;Zt)- 

Bougerol [3] gave conditions of Lyapunov type for the existence of a stationary solution to Sto- 
chastic Recurrence Equations (SRE), which are particular cases of 111. 2ft . 



Other approaches than (jl.ip to modeling processes which do not satisfy the Markov property al- 
ready exist; the Random Systems with Complete Connections (RSCC), see [2TJ, and the Variable 
Length Markov Chains (VLMC), see [3]. Such models are widely used in the fields of particle 
systems or in DNA data analysis. These processes are defined through their conditional distribu- 
tions. Their existence relies on assumptions on the conditional expectations, following the work of 
Dobrushin [10]. Notice that Berbee [2] obtained another existence condition for the cases where 
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the state space is discrete, see also [5j [18]. 



Dobrushin's condition implies mixing, see [12\ I21j . Mixing coefficients are useful to derive as- 
ymptotic theorems for various functionals of a stationary sequence, see Rio |30| . However, major 
asymptotic results still hold under so-called weak dependence conditions, see section 12.21 [HI E3] 
and the recent monograph by [Tj- The Central Limit Theorem (CLT) in Dedecker and Doukhan 
|6j holds if the x 2 ln(l + a;)th moments of Xq are finite and if the process is weakly dependent with 
geometric decay of the coefficients. Because weak dependence is less restrictive than mixing (see 
Andrews pQ for an example) this result extends the CLT for mixing sequences due to Rio [30] . 
The conditions for the CLT are expressed in terms of Orlicz functions that balance the moments 
of some order and the weak dependence conditions. 

The existence of a stationary solution to (II. ip is proved in section 13.21 under a specific Lipschitz 
type assumption on F, see (13. ip below. Approximation by suitable Markov processes is the main 
tool for the proofs given in section This existence condition also yields finiteness of moments 
of some order in terms of Orlicz functions. We get bounds for the weak dependence coefficients of 
the solution to ([Lip , We use these bounds to derive sufficient conditions on F in term of Orlicz 
functions and in turn to prove a Strong Law of Large Numbers (SLLN), a CLT and a Strong 
Invariance Principle (SIP), see section [3T3l We discuss the generality of our model in section [4] 
comparing it with existing ones. But to begin with, we introduce some notation and we define 
useful tools such as weak dependence and Orlicz spaces. 



2. Preliminaries 



2.1. Notation. In the sequel, the iid innovations £t for t G Z take values in a measurable space 
(E',A r ). Let || • || denote the norm of a Banach space E. The space is the subset of E n 

of finitely-non-zero sequences (xk)k>o such that there exists N > with xt = for k > N. Let 
E be endowed with its Borel a— algebra A, then E^°°^ is considered together with its product 
a— algebra A®^ . The function F in (jl.ip is assumed to be a measurable function from E\°°> x E' 



X 



I = ^\\X\\ m for m ^ 1 for 

su PxGE 1^0)1 and 



with value in E. Moreover || • || m denotes the usual L m -norm, i.e., 
every E- valued random variable X. For h : E —* M, we denote \\h\ 

r- \h{x)-h{y)\ 
Lip(/i) = sup f 21 . 

The space Ai(.E) is the set of functions h : E — > M such that Lip(/i) ^ 1. 

2.2. Weak dependence. An appropriate notion of weak dependence for the model (jl.ip was 
introduced in [B]. It is based on the concept of the coefficient r defined below. Let (f2,C,P) be 
a probability space, M a c-subalgebra of C and X a random variable with values in E. Assume 
that \\X ||i < oo and define the coefficient r as 



t(M,X) 



sup 



f(x)P xlM (dx) - J f(x)P x (dx) 
An easy way to bound this coefficient is based on a coupling argument: 

t(m,x) < pr-y||i 



with / € Ai (E) 
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for any Y with the same distribution as X and independent of A4, see [8]. Moreover, if the 
probability space (f2,C,P) is rich enough (we always assume so in the sequel) there exists an X* 
such that t(M.,X) = \\X — X*\\i. Using the definition of r, the dependence between the past 
of the sequence (X t )t£Z and its future /c-tuples may be assessed: Consider the norm \\x — y\\ = 
— Ui\\ + • • • + ||^fc — Vk\\ on E k , set M. p = a(X t ,t ^ p) and define 

T k (r) = max T sup{r(A4 p ,(Xj 1 ,...,X 7 )) with p + r^ii <••• <^), 

Too{r) = sup r fc (r). 
fc>0 

For the sake of simplicity, ^(r) is denoted by r(r). Finally, the time series (Xt)t£Z is r-weakly 
dependent when its coefficients r(r) tend to as r tends to infinity. 



2.3. Orlicz spaces. Orlicz spaces are convenient generalizations of the classical L m -spaces, we 
refer to [24] for the introduction and properties of such spaces. Let <3? be an Orlicz function, i.e., 
defined on M + , convex, increasing and satisfying ^(O) = 0. For any random variable X with 
values in E, the norm \\X\\$ is defined by the equation 

11X11$ = inf \u > with E$ ( ^—^ ) ^ 1 



The Orlicz space L* is given by 

L* = | E- valued random variables X such that ||^||$ < oo}. 

It is a Banach space equipped with the norm || • ||$. For m ^ 1 and <£(x) = x m , notice that L* 
is the usual L m -space. We restrict ourselves to Orlicz functions <3? satisfying the condition: 

(2.1) For all x,y G R + , $(xy) ^ $(x)$(y). 

This class of Orlicz functions is sufficiently large. For instance, the functions <I>(x) = x m and 
&(x) = x m (l + ln(l + x)) m ' satisfy (|2.ip for each m ^ l,m' ^ 0. Moreover, if eft is any Or- 
licz function satisfying the A2-condition (there exists k > such that (p(2x) ^ k(j){x)) then 
<fr(x) = sup u>0 4>(xu) / 4>(u) is an Orlicz function satisfying (|2.ip . Various examples of Orlicz func- 
tions satisfying the A2-condition are given in 



Later, in theorem !3.2l we will need some transformations of Orlicz functions. Given such a function 
<]?, we define for q > 1, 



(2.2) = sup^xy)"- 1 - $(y)/y}. 

y>0 



The transformations & q {x) have simple bounds for certain choices of see lemma [57T1 for details. 
In particular, if $(x) = x m for m > g > 1, then $ g (x) ^ l)(g-l)/(m-g)_ Another useful 
example is the one of <J?(x) = x q (l + ln(l + x)) ( - 1+b ^ q ~ 1 ^> and 3> 9 (x) ^ exp((g - l)x l /( x+h })x q ~ 1 for 
any q > 1 and 6^0. 
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3. The results 

3.1. Assumptions. The existence of a solution to (jl.ip will be proved under a Lipschitz-type 
condition. We express it in terms of some Orlicz functions in order to be able to work with 
moments more general than power moments, see theorem 13.11 These moments will be needed to 
establish the asymptotic results of theorem 13.21 

Assume there exists an Orlicz function such that for all x, y in 

oo 

(3-1) \\F(x;^)-F(y;^)U^J2 a ^ x j-y^ 
where (aj)j^i is a sequence of nonnegative real numbers such that 

oo 

(3.2) a = a j < 1 an d 

(3.3) = 11^(0,0,... ;£o)||* < oo. 

The Lipschitz property of F and the moment assumption (|3.3h induce that \\F(c; £o)||<£ < 00 f° r 
any constant c S E^ 00 ^. We choose c = (0,0,.. .) in condition ()3. 3f) for convenience. 

3.2. Existence, moments and weak dependence. The following theorem settles the existence 
of a solution to (jl.ip . It also states that the <J>th moment of this solution is finite. 



Theorem 3.1. Assume that conditions (|3.2p and (|3.3p hold for some Orlicz function & satisfying 
(|2.ip . Then there exists a r -weakly dependent stationary solution (X t )tei °f fll-ip such that 
||Xo||$ < oo and 

r(r) ^ 2-^— inf I a r/p H — a k i — u «.s /•— x. 

\ fc=p+i 

The proof of the existence of a solution to (jl.ip is given in section 15.31 expressing it as the limit of 
the p-Markov processes defined in (|1.2|) . The weak dependence properties are proved in section 

Remark 3.1. We also prove in section [5] that there exists some measurable function H such 
that X t = H(^ t ,Ct-i, ■ ■ ■)■ This means that the process (X t )t<=z can be represented as a causal 
Bernoulli shift. For those processes, conditions (|3.2p and (|3.3p together imply the Dobrushin 
uniqueness condition, see [ID] . Thus (Xt)tez is the unique causal Bernoulli shift solution to (jl.ip . 
Moreover, as a causal Bernoulli shift, the solution (X-t) te z is automatically an ergodic process. 
Under the conditions of theorem 13.11 the solution to (jl.ip has finite ^th moment. From lemma 
15.31 (Xt)tcz has also finite first order moments. The ergodic theorem yields the SLLN for any 
chain with infinite memory under the assumptions of theorem 13.11 

Corollary 3.1. Under the assumptions of theorem \3.1\ there exists a r-weakly dependent sta- 
tionary solution (Xt)t<=z to (|1.2p such that ||Xo||$ < oo and r(r) ^ 2^1(1 — a) -1 a r ^ p for r ^ p. 

Dedecker and Prieur [8] proved the existence of a solution to (jl.2l) . They stated that there exists 
< p < 1 and C > such that r(r) ^ Cp r . Applying corollary 13. 1\ we get the bound p ^ a 1 ^ p . 
The bounds of the weak dependence coefficients in theorem 13. II come from an approximation with 
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Markov chains of order p and from the result of corollary 13.11 

In theorem 13. 1\ the r-weak dependence property is linked to the choice of the parameter p and 
then to the rate of decay of the Lipschitz coefficients dj. For example, if aj ^ ce - ' 3 - 7 , we choose 

p as the largest integer smaller than y 7 — \n{a)r/ f3 to derive the bound r(r) ^ (7 e ^v / ~ ln ( a )/ 3r 
for some suitable constant C > 0. If aj ^ cj~@, we choose the largest integer p such that 
plnp(l — /3)/lna ^ r. Then there exists C > such that r(r) Cp 1 ^^. Notice that lnr is 
smaller than lnp + lnlnp up to a constant and that lnr/r is proportional to l/p(l +lnlnp/lnp) 
and then equivalent to 1/p as p tends to infinity with r. From these equivalences, we achieve thus 
there exists C > such that r(r) si C (In r/r)^~ l . 

A similar result as the one of theorem 13.11 was obtained for discrete state space models (as RSCC) 
in [20]. They gave bounds for the mixing coefficients under conditions on the marginal distribu- 
tions of the innovations. The bound in [20], theorem 2.1.5 on page 42, is similar to the one for 
r(r) in theorem 13.11 In a sense we extend their result: Here the innovations are not supposed 
to be absolutely continuous and our approach can be applied to discrete state space processes as 
well, see the example of the Galton- Watson process with immigration in section [U 

Bougerol gives in [3] a recursive approximation of the stationary measure in the Markovian case. 
In proposition ^. ll below we generalize this result to the infinite memory case. Let 4>k ■ E k ~ 1 xE' — > 
E be the random function defined as x i— > F(x,c;^k), for each k ^ 2 and some fixed sequence 
c = (ci, C2, . . .) & E(°°\ Write X\ = <j>i = (f>(c; £o) an d define recursively 

X n = (j) n (X n -i, . . . , X\). 

Proposition 3.1. Assume that conditions (|3.2p and f)3.3|) hold for satisfying (12. ip . If (X t ) t& z 
is the solution to (jl.ip then 

\\X r — XJL ^ - — ^JjLjlZ mf | a r / p H | — > as r —* oo, 

1 — a l^p^r I 1 — a z — ' ; 

where c is a constant such that ||cj|| ^ c /or all i ^ 1. 
The proof of this proposition is given in section 15.61 



3.3. Asymptotic results. In this section, E = R. We give an appropriate condition on F (see 
()3.4p below) that leads to versions of the results of Dedecker and Doukhan [6] and Dedecker and 
Prieur [8] obtained under weak dependence. 



Theorem 3.2. Assume that conditions (13.21) and (13.3P hold for some Orlicz function <3? satisfying 
(12. ip and assume there exists cq > suc/t £/iat 

(3.4a) a^ q (cQk) < oo if there exist p ^ 1 such that = 0, 

(3.4b) ^^^fc^g I — cpfcln ( J < oo otherwise, 

where Q q is defined in (12. 2D . T/ie following relations hold: 
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SLLN: J/g€]l,2[ then ^^^(I, -EI ) 0, 



a.s. 
i=i 



1 [nt] 

CLT: Ifq = 2, then — ^(Xj - EX ) -U J (rW(t) asn^oo 



in 

00 



where a 2 = Cov(Xo,Xi) is finite andW(t) is the standard Wiener process. 

i=— 00 

SIP: If q = 2 and if the underlying probability space is rich enough then there exist inde- 
pendent jV(0, a 2 ) -distributed random variables (Yi)i^i such that 



~^2(Xi — Yi) = o(Vn lnlnnj a.s. 
i=l 

The proof of this theorem is given in section 15.51 



Note that x 2 ln(l + x)th moments are necessary to get the CLT for weakly dependent processes. 
See |15| for an example of processes, solutions of (II. 2D for p = 1, that do not satisfy the CLT 
under conditions (|3.2[) and (|3.3I) for $>(x) = x 2 . Note also that approximations by martingale 
difference as in [28] or projective criterion as in [27] give the CLT under weaker assumptions for 
some of the examples treated in section HI 

Condition (|3.4aj) is relevant for the Markov solution (X t )t<^z to (|1.2p . i.e., when Ylj>p a i = 0- 
For the other cases, we rewrite assumption (I3.4bp for various rates of decay of the Lipschitz 
coefficients aj. Let a, b, c be some positive real numbers then 

(]3.4af ) If afc ^ ck~ a , ^afc<I>g (coklnk) < 00 for some cq > 0. 

(|3.4ar ) If dfc ^ cexp(— ak b ), ^^Ofc^g (cok 1+b ^j < 00 for some Co > 0. 

fc^l 

For instance, condition (|3.4a[p holds if $>(x) = x m for m > q and a > 1 + ((7 — l)(m — l)(m — g) . 
Condition (l3~laTT) holds for $(x) = x q (l + ln(l + x))( 1+6 ^ 9_1 ). Applying theorem ESQ the CLT 
and the SIP hold for sub-geometric rates of decay of the Lipschitz coefficients as in (|3.4arp under 
a moment condition of the order x 2 (l + ln(l + x)) 1+b . 

4. Examples 

In this section, we present some examples with E = R d and d ^ 1. We consider the finite 
memory case as well as an infinite memory extension of Stochastic Recurrence Equations (SRE). 
In particular, we consider the example of the Galton- Watson process with immigration which 
satisfies the conditions of our results, but it is not a SRE in the sense of [3]. 

4.1. Markov models. 

SRE. We consider an iid process (4>t)t& of random Lipschitz maps with 1 1 0t (a^i ) — 
<Myi)|| ^ L{<f))\\xi — yi 1 1 a.s. for all x\,y\ 6 E and t 6 Z. Moreover let 4>t(x) be 
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measurable for every fixed x 6 E and t 6 Z, If a stochastic process (XtjteL with values 
in E satisfies the equation 

X t+ \ = <pt(Xt) a.s., for all t € Z, 

we say that (X t )t G z obeys the SRE associated with ((j) t )t&- We write this equation as in 
(|1.2p setting = 0t for t G Z, and F(x,z) = z(x) for x £ E and z £ the space of 
Lipschitz random functions. In this case, conditions ()3.2f) and ()3.3f) become 

\\L^)U<1 and ||0o(O)||*<oo. 

Weaker conditions related to a Lyapunov exponent for the existence of an a.s. solution to 
a SRE are obtained in [3]. However, that result does not yield the existence of moments 
nor asymptotic results as those in theorem 13.21 We also mention the survey article by [9] 
for an overview and other nice application of SREs. 

Nonlinear autoregressive models. Here we consider a solution to (jl.lj) . where E' = E 
and F admits the representation 

F(xi, ...,x p ;s) = R(xi, ... ,x p ) + s. 

Condition (13.21) becomes 



\\R(yi, ■■■,y P ) - R{xi,...,x p )\\ < ^ajWxj -yj\\ with ^ aj < 1, 

3=1 3=1 

and condition (|3.3p coincides with ||£o||$ < °o- Results similar to those in theorem | 
are obtained by different methods in |17j . 

Galton- Watson processes with immigration. If E = R, a Galton-Watson process 
with immigration is given as a stationary solution of the equation 

£ Cm + Ct, if X t _i > 0, 



(4.1) X t 



if = 0. 



Here ((t,i)t<az,i>o, (Ct)tez are independent iid families of integer-valued random variables 
and E' = N N is equipped with the product measure. We can write X t = F(X t -i,^t) with 
F(x, (uj)j^o) = uo + Th=i u i if x > and ^(0> (, u i)i^o) = for any (u^q. If y% > x x > 
then F(x, (^)»^o) - ^(^, («i)i>o) = Si= Xi u * tnus 

yi 

\\F{x\,£o) - F(yi,£ )h = || ^2 ^lU = ' yi ~ 1 1 1 Co,o 1 1 <a> - 

i=xi 

Assumptions ()3.2p and (|3.3[) hold as soon as ||Co,o||* < 1- This model is not a SRE if 
Co,o is not finitely supported, thus we are not under the conditions of (3J- Other non-SRE 
examples which can be treated by our approach are given in [26 . 

4.2. SRE with infinite memory. Infinite memory extensions of classical SRE are solutions of 
the equation 

J Xt = </>t(Xt-i,X t -2,'--) a.s., 

1 \\M x ) - My)\\ < Ei^i L i(<^)lki — s/*IU a - s - 
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for all x = (xi)i^i,y = {yi)i^\ G E^°°\ Here ((p t )t^z is an iid process of random Lipschitz maps. 
If X^i>i 11-^(^)11* < 1 then conditions (|3,2p and (|3.3p are satisfied. Some examples with this 
representation follow. 

Non Linear ARCH(oo) models. Here (Xt)t£Z is the stationary solution of the equation 

X t = £t (a + f^a^X^ , 

where & is a d X k matrix, E' = Mk,d(^), a G M fc and aj : E —* R k are Lipschitz 
functions. The LARCH(oo) model of [19j [16] corresponds to the special case of linear 
functions ctj(x) = CjX with k x d matrices Cj. Assumptions (|3.2p and (|3.3p hold as soon 
as llCo||*X^i Li P Q .j < 1 and Z^i a j(°) < 00 . 

Models with linear input. Let / : M fe x E' — > E be measurable and satisfy \\f(t, £o) — 
Co) II* ^ ^ P — s|| for some finite constant L > 0. We consider 

oo 

i=i 

where Cj are k x d matrices. Relations p.2p and (|3.3p hold if £^j>i ll c ill < 1 an d 
/(0, Co) ||* < °°- These models are used in statistical mechanics, see [22] , 
AfRne models. Let us consider the special case of chains with infinite memory that can 
be written in a bilinear form 



(4.2) X t = + f t , 

where Mt = M(Xt-i,Xt-2, ■ ■ •) and ft = f(Xt-i,Xt-2, ■ ■ •) are both Lipschitz functions 
of the past values X t -\, X t -2, X t -3, .... Applying theorem 13.11 under the condition 

oo oo 
i=l i=l 

there exists a weakly dependent solution to ()4.2p . This class contains various time series 
models (such as ARCH, GARCH, ARMA, ARMA-GARCH, etc.). In the appendix we 
prove the existence of the joint densities of the solution to (|4.2p . This result and the weak 
dependence properties obtained in theorem 13.11 are needed for achieving optimal rates of 
convergence of nonparametric estimators, see |29j. 



5. Proofs of the main results 

After some preliminaries in section 15. 1\ in section 15.21 we construct a solution of the Markov 
model (jl.2p . We use it to approximate the solution to (|1.1|) . The existence of a solution to (jl.ip , 
presented in theorem 13.11 is obtained as p — > oo in section 15.31 Its weak dependence properties 
are derived by coupling techniques in section 15.41 Using weak dependence results of [5] , we 
prove theorem 13.21 in section 15.51 Finally, we derive the proof of proposition 13.11 in section 15.61 



WEAKLY DEPENDENT CHAINS WITH INFINITE MEMORY 9 

5.1. Preliminaries. We first present four useful lemmas. The first one aims at bounding the 
transformations & q for q > 1, the other ones are used in the proof of the existence of a solution 

of CEO). 

Lemma 5.1. Assume L is an increasing non-negative function on [0, oo] and write L^ 1 for the 
generalized inverse of L, i.e., L^ 1 (x) = inf{y > with L(y) ^ x}. If = x q L(x), x ^ 0, 
for some q > 1 then 

$ q (x) < (xL-\x q - 1 )) q ^ 1 for all x > 0, 

Proof. From (12. 2j) . we have & q {x) = sup^ >0 {y' ?_1 (x' ? ~ 1 — L(y))}. We restrict ourselves to y ^ 
L _1 (x 9_1 ) otherwise y 9_1 (x 9_1 — L(y)) ^ 0. Now notice that the first term of the product 
y q ~ 1 (x q ~ 1 — L(y) is increasing and the second term always remains smaller than This 
proves the lemma. □ 



Lemma 5.2. Assume that the Orlicz function $ satisfies (|2.ip . Let £ and £ be independent 
random variables, z a measurable function and Z = z(£, £)■ We write for the expectation with 
respect to the distribution oft;. Define 

(5.1) IKCOIk^ =inf {u > with E^(\\z(s, ()\\/u) sC 1} . 

Then \\Z\\$ ^ II ll^ll*,? ||$ • 

Proof. One needs to prove that E$(Z/|| ||Z||$^||<j,) ^ 1: 



^Il$cll4 / \II-^II<E>,£ llll^ll$,^ll^ > 



Z \ ( \\z\ 

$ 77=7, $ 



^11*,?/ V llll-ZII^H* 



The last inequality follows from (|2.ip . By independence of £ and £ and by (|5.1 



M> ! „ f ) ■ ; E 
l^ll$,dl* 



\Z\ 



1^1 



We conclude by using the definition of the norm [| • ||$. □ 



Lemma 5.3. /f the Orlicz function $ satisfies (12. ip f/ien /or any E-valued random variable X 
we have \\X\\i ^ ll^ll*- 

Proof. Using Jensen's inequality, we obtain 

Note that $(1) < $(1) 2 by (J2HD and then that $(1) ^ 1. We conclude that ||X||i < \\X\\$ by 
using the definition of the norm || • ||$. □ 

Lemma 5.4. Let uq ^ and (« n )neZ ^ e a rea ^ sequence such that \u n \ ^ uq z/n < 0. Assume 
that 

v 

(5.2) u n = aiu n -i, Vn > 0, 

i=i 

where a±, . . . , a p are fixed nonnegative numbers with a = Y^a=i a « < 1- Then, 

supufc ^ a n l p UQ, Vn ^ 0. 
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Proof. By a recursion argument, one first shows that sup k ^ n Uk ^ uq. Then (u n )„ g N is bounded 
by «o- Let v n = sup fc ^, n Uk for n G Z. Using the relation (15 2|) . we get u n ^ av n - p for all n ^ 0. 
Then recursively u n ^ « _ '~ Tl//p ^n+p[-n/p]- From |u n | ^ no if n < 0, v n+p i n /p\ = vq = uq because 
n +p[—n/p] 0. The result follows from — [— n/p] n/p. □ 



5.2. p-Markov stationary approximations. In order to construct a solution to (jl.ip we con- 
sider, for each fixed p ^ and q > 0, the p-Markov process (X p>q ^)t^o defined by X PtQ j = for 
t ^ —q and the recurrence equation 



if 



t > q. 



(5-3) -^p,g,t — F(X P) qj;-i, • • • , X Pt q t t-p, 0, 0, ... ; 

Using the notation of lemma 15. II with £ = £o and £ = ^p,q,-2, • • •) andz(£,C) = i ? (£,£), 

the Lipschitz condition (13.11) implies that 



|^p,<j+l,0 — -<^p,<3,o||<I>,£ ^ '^2 a i\\Xp ! g+l-i — X, 



P,Q,-i\ 



i=l 



Applying lemma I5T2 



|^p,?+l,o — ^35,9,0 1|$ ^ 



||^p,g+l,0 — Xp,q,o\\§,£ ||$ 
P 

^] a « 11^,9+1,-4 — X 



1=1 



^ a * H-^P>9+b— i ~~ X p . 



i=l 
P 

i=l 



The last inequality follows from the fact that by the definition of X Pi q—i and Xp >q —io, these 
quantities have the same law for each triplet of positive integers (p,q,i). We now consider 

v n = ||-Xp,n+l,0 ~~ X r , 



L p,n,o||$ for n G Z, with n 



if n < 0. For n > 



i=l 

From lemma [53] we obtain 

*S a n/ % ^ a^||X P)li0 || $ < a«^||F(0,0,...;6)lk < « n/ V$- 

Hence, for each p, (X P;nj o)neN is a Cauchy sequence in L*; it converges to some X pfi e L*. From 
its construction, it is clear that X Pjn) o is measurable with respect to the a— algebra generated by 
^ 0}. The L*-convergence ensures that this is also the case for X Pj o- Hence there exists 
some measurable function H p such that X p .o = H p (^q, . . .). As n f 00, a continuity argument 
on F implies that X Pj o = F(X Pt -i, . . . , X p _ p , 0, 0, ... ; £0) and shifting the lag t £ Z leads to the 
equalities, 

X Pi t = ilp(£t,£t_i,£t_2, • • •) = F[X Pt t-i, ... , X P: t- p , 0, 0, ... ; 
Then the sequence (X P) t)tez is a stationary solution of the recurrence equation (|5.3p for each 
P > 0. 
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Consider 

M*,p = ||-Xp,t||f>) ^*,P,t = II^P+l,t ~~ -^p,t||$! 

The definition of fi$ tP given here for p > extends to p = since Xo t t = F(0, 0, . . . ; £t) satisfies 
||^o,t||* = by e q n - (USD- 



Lemma 5.5. Assume conditions (13.2[) and (13. 3|) hold for some Orlicz function <3? satisfying (12.1 
T/ien 

A A M* 

^*,oo = sup/i$ iP ^ and A* p = supA 4 p t ^ a p+ i — 

p>o 1 - a t & \\ — aj 

Proof. From eqn. (|3.2p . we have that 



M*,p ^ ll-^p,* ~~ -^o.tll* + A 1 * ^ a jll^p,t-j II* + ^ ^<i>,p a j + A 1 *! 

hence ^ (1 — a) _1 ^x$ and ^$ j00 ^ (1 — a)~V* follow. In a similar way, we obtain the 
inequalities 

^<s>,P,t = |-F(-Xp+i,t-i> • • • > ^p+i,t-p-i 5 0, 0, . . . ; £t) — ^(-^p,t-l> • • • )-Xp,t-p)0, 0, . . . ;£t) 
p 

^ ^2 a 3 : \\X P +i,t-j — Xp,t—j II* + a p+i||^p+i,t-p-i||<i> 
p 

This implies that A$ iP ^ a p +i(l — a) -1 /^^! and the result of lemma [531 is shown. □ 

5.3. Proof of the existence of a solution to (jl.ip . Note first that lemma 1531 implies that 
X Pi t -^ P ->oo Xt in L* since this space is complete. The continuity of F ensures that X t is 
a solution of eqn. (jl.ip . Furthermore, as a limit in L* of strictly stationary processes, Xt is 
also stationary (in law) and ||.Xt||$ < oo. Finally, Xt = #(£t, £t— 1> ■ ■ ■) is the limit in L* of 

X pi t = H p (£t, 6-1; • • •)• 

5.4. Proof of the weak dependence properties. The weak dependence property of a solution 
to (jl.ip is formulated in terms of the L 1 -norm in the definition of the coefficients r. As shown 
in lemma [531 ||^||i ^ 11^ II* f° r an Y F- valued random variable X. Then assumptions (|3,2p and 
(|3.3p are always satisfied replacing || • ||$ with || • ||i. We first prove corollary 13.11 



XI 



Proof. We use coupling techniques to evaluate the coefficients t, see pj2j Let (Ct)t£Z be an 
independent copy of (£,t)t<=z- We define the process (X* t ) t ^z as 

r f{x;^ x; >t _ p , o, o, . . . ; for t < o ; 

l ^ \ F(X; t t _ l7 . . . , X* t _ p , 0, 0, . . . ; 6), for f > 0; 

Using similar arguments as section 15.21 there exists a sequence of measurable variables with 
respect to the a— algebra generated by £' t ,t ^ denoted by (X* n0 ) n ^ such that it converges in 
L*toX* G L*. TheL* -convergence ensures that X* g are also measurable variables with respect 
to the o~ — algebra generated by ^ 0. Then, by definition of t ^ 0, X* Q is independent of 
Xpfl. If there exists a non-increasing function 6 p (r) of r such that ||X Pjr — ^ S p (r), we have 
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Tpr ^ $p( r )- This follows from the coupling property of weak dependence coefficients r explained 
in [8]. 

Assumption (|3,2p and lemma [531 yield 



I Y — Y* I 
\ yv p,r - ri -p,r\ 



Y* I 
p,r-i ~ ^p t r— » 1 1 X - 



j=l 



Denoting u> r = ||X P)T .— X* ||i for r G Z, we again use lemma l5~il and the relation ||.F(0, 0, . . . ; £o)||i 
;Ui to obtain 



it', 



a r ' p w ^ 2 fH a r/p ^ 2-^—a r/p . 

I — a 



Now choose 5 p (r) := 2fi\(l — a) l a r / p leads to the result of corollary 13.11 □ 
Now we finish the proof of theorem l3.1l defining the process (X^)t£z as the solution of the equations 



X 



F(X t U,X* t _ 2 ,...;e t ), fort^O; 
F(Xt_ lf Xt_ 2 ,...;£ t ), fori>0; 



We remark that (X£)t is also a stationary chain with infinite memory. Lemma 15.51 gives 



Xr — X' 



p,r ||l 



k=p k=p 

The same bound holds for the quantity ||X* — X* r ||i. For each integer p, 



I V V*ll ^ II V V li II v II I || v"* V* II o ^ 

|A r — A r ||l ^ ||A r — A Pjr ||i + ||A Pir — A pjJ ,||i -f- ||A r — A pir ||i ^ Z- — 



Ml_ r/p 



+ Ei^ • 

fc=p+i 



Because this bound is non-increasing with r, we conclude the weak dependence properties in 
theorem 13. II by using the coupling technique. 



5.5. Proof of theorem 13.21 First we recall the assumption (D(q) ) of [6 J for q > 1, 

l^olli 



(D(q)) 



o G(u)du < oo, 



where (t/2) _1 (u) = inf{A; G N/ r(k) ^ 2u}. Here Q denotes the generalized inverse of the tail 
function x i— > P(|A"o| > x) and G the inverse of x Q(u)du. Dedecker and Doukhan proved 

in [6] the SLLN and the CLT under (D(q)) for respectively 1 < q < 2 and q = 2. The SIP is 
proved in [8] under (D(q)) for q = 2. Write A(p) = y^j> p aj and A -1 its generalized inverse 
A-^u) = inf{k G N/ A(u) ^ u}, 

V q (x) = ^(x 1 /^ 1 )) /x 1/{q - 1] and %{x) = sup{xy - tf,(y)}. 

Noticing that A' 1 ^) = k on ] J 4(/c - 1); A(k)] and that $ g (x) = ^(x 9-1 ), there exists C > 
such that 

I ^ q {c Q {A- l {u)-l)\n{u))du^cY,^ q c A:(l- l {E ^ fc% . >0 }ln(^a J )) . 

J ° k^l \ j^k J 
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Then assumption (|3.4p implies that we work under the condition 
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(5.4) 



((co(A-V)-l)ln(u)) 



q-l 



du < oo. 



We want to prove that condition (15. 4h implies (D(q) ) for all q > 1. The first step is to prove the 
bound 



(5.5) 



(t/2)-\u) < 



A 



-i 



1 -a 

1 

2mi 



In a 



a fc/p + A(p + 1) ) < u \ . 



Theorem 13.11 gives (r/2) (u) ^ inf B with 

£ = J k G N such that 3p ^ 1 with 

[ 1 — a 

Set f = (1 — a)(2/ii) u, the integer p* = A _1 (f) — 1 is close to the infimum of B. Then all 
integers k with a fc//p ?J v belong to B, as for instance k* = [(A _1 (?;) — l)lnu/lna] which is 
then larger than (t/2)~ 1 (u) by definition. Observe that A~ l {y) = 1 as soon as v ^ a, thus 
[(A- 1 ^) - l)lnu/lna] = for v ^ a. 



Using this estimate of (r/2) in (|5.5p . condition (D(q) ) holds if 



(5.6) 



(A -1 (u) - l)lnu 



In a 



p-i 



Q p ~ l o G 



1-a 



Let \I/ be an Orlicz function and ^*(x) = sup y>0 {xy — ^(y)} be its Young dual function. For any 
functions / and g, Young's inequality gives: 



f(x)g(x)dx < 2 inf <^ c > with / ^ 



/to 



dx < 1 



x inf < c > with 



" vl/ ' f ^- | ,/,■ < 1 



In the following we apply this inequality with and ^ = K^ p for some K > 0, f(x) = Q p 1 o 
G(2/ii(l - a) _1 x) and 5(2;) ^ ((A _1 (x) - 1) ln(l/x)(- In a)" 1 ) 9 " 1 . Note that the Young dual 
function is here ty*(x) = K^*(x/ K) and then f£ f(x)g(x)dx is equal to the left hand side term 
(|5,6p up to the choice of the constant K > 0, see below. In view of Young's inequality, the first 
term in the bound of (|5.6p thus expresses as the infimum over c > such that 

ll* " 1 $(QoG(m) /c) 



K- 



A-a 



QoG(u) /c 

Replacing G(u) with x, one obtains the simpler inequality: 



-du sS 1. 



1-q 



1 - a 

cdx = K cE<3? 

2mi 



^ 1. 



The last equality is set using the definition of Q(x). If assumption ()3.3[) holds, the last inequality 
is satisfied for K = 2/Ui/iJ 1 and c = /i$(l — a) -1 . 
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The second term of the Young inequality expresses as the infimum over c > such that 



(5.7) 



K / W r 



K(-\na)i- 1 c 



dx < 1. 



Because & q {x) = ^*(x q x ) we check that 



g 3> g {cq(A-\u) - 1) ln(n)) du V 1 = 

(K A 1)(— In a) p_1 ' ° X 

satisfy the relation (|5.7p . It is obvious by (|5.4p that c\ < oo and then we proved the implications 

(EU|) with > 1 => ((53]) with q > 1 => ( |D(q)[ ). 

This ends the proof as the results of theorem 13,21 are versions of the results in [61 [8] that hold 
under assumption (D(q)). 

5.6. Proof of proposition 13.11 Let n be a fixed integer and s n ^ n — 1. Let (JT^ez be the 
stationary solution of X t = F(X t -i, X t -2, 0, 0, . . . The Lipschitz assumption (|3.ip implies 
for 1 ^ k ^ n 

fe-1 



-Xfc — Xk 



^ fli -^*-» ~~ -^fc-i + ^ a>i\\Xo — Ci 



i=i 



i^k 



The sequence = X^+i — -X"fc+i , A; = 1, 2, . . . satisfies the recursion 

11$ 

k 

v k ^ a j v k-j + "Wfe f° r an k ^ 1 

where = (\\Xq\\§ + c)Y^ l j > k a j f° r k ^ 1. Notice that Ik^oo 0. We first prove the bound- 
edness of (vk)keN- Let £ be a fixed integer. For all A; such that £ ^ k, ^ asup^Vi + ui. We 
deduce that supj^fj ^ iti. Finally ||u||oo ^ a ll^o||<i>/(l — a). 
Now for all integers k, s ^ 1 such that £ ^ fc + s, 

k l oo 

< y~] + a j v e-j + u e ^ asupvj + ||w||oo ^ ctj + ""fc+s- 

j=l j=k+l ^ s j=k+l 

This inequality holds for all £ ^ k + s. Then 



sup t>j ^ asupfj + |H|oo a j +"Ufc- 



We deduce that 



SUp Vj ^ ^Hulloo + 



I- a 



j=k+l 



\v\\oo ^2 a 3 + Uk 
j=k+l 



Using the inequality ||v||oo ^ a ll^o||<i>/(l — a), one gets the result. 

APPENDIX 

We give below general conditions for the existence and the boundedness of joint densities of Affine 
Models defined in section SJ Thus we extend the results for Bilinear Models given in 
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Proposition 5.1 (Regularity of affine models). Here E = E' = M. d for some d ^ 1. Suppose 
that the innovations (£t)teZ in the model (|4.2p have a common bounded marginal density fg. 
Moreover, if inf^ ) det M((xj)j>o) = M_ > 0, the marginal densities fx lt ...,X„ of (X±, . . . , X n ) 
exist for all n > and satisfy 

Proof. The solution X( = H(£t,£t—l, ■ ■ •) obtained in section [5\3l is independent of {Cj)j>t- If Gr 
is a bounded continuous function on E with value in R, it holds that 

EG(Xt) = EG 1 (M(X ,...)C 1 + f(X Q ,X_ 1 ,...)) 




Gi(M(«)a a + /(«)) / € (s 1 )d Sl P (XoiX _ 1 ,... ) (d«) 

^ Ml J G(x 1 )/ f (M- 1 (n)(x 1 -/(n)))P (XojX _ li ... ) ( ( in)d Sl . 

The last inequality follows by the substitution M{u)s\ + f(u) = x\ valid under the assumption 
inf( a ..). >0 det M((xj)j>o) = M > ensuring that M(u) is invertible for all u. We obtain 

fxM ^ JMT 1 J f^M-\u)( Xl -/(«)))P(x ,x_ 1 ,..)(rf«) < M-'ll/slloo. 

We proceed by induction for the cases n ^ 2. Assume that ||/xi,...,X„_i ||oo ^ M"^ 1 ) || f^H^T 1 is 
satisfied. Let G n be a bounded continuous function on E n with value in R, one has 

EG n (Xi , . . . , X n ) = KG n (Xi, . . . , X n -i,M(X n -i,X n -2, . . .)£n + f(X n -i, X n -2, ■ • •)) 
G n (xi, . . . ,x n -i,M(x n -i, . . .,xi,u)s n + /(x n _i, . . .,x 1 ,u)) 

ft(s n )ds n f(x u ...,x n -i)( x i, ■ ■ -,x n ^x)dxi ■ ■ ■ dx n -idP( Xo ,X- 1 ,...\x u ...,x n - 1 ){ u )- 
The substitution M(x n _i, . . . , x\,u)s n + /(x n _i, ... ,xi,u) = x n yields 

fx lt ...,x n (xi, ...,x n ) ^MjJ f^{M~ 1 {x n -x,.. .,x 1 ,u)(x n - f(x n -i, . . .,x 1 ,u))) 

f(Xi,...,x n ^i){ x i ) • • .,x n -.x)dx\ ■ ■ ■dx n -. 1 dP( Xo ,X- 1 ,...\x 1 ,...,x n - 1 )(u). 

Together with the induction assumption \\fx lt ...,x n -i II do ^ M~ 11/gllST 1 ) this last inequality 
yields ||/x 1 ,...,xJ|oo^M-"||/ c ||S . ^' ' □ 
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